Frame Rate Stabilization for Real-Time Free-viewpoint Video Generation -3D Shape Reconstruction in Multiple Resolution-

This paper represents a method to stabilize the frame rate of real-time free-viewpoint video generation. When we apply real-time free-viewpoint video to live video broadcasting, the frame rate of video processing should be constant. Otherwise, presented video gives viewers unnatural feeling due to time inconsistency. Our method to generate freeviewpoint video is based on 3-D shape reconstruction and visualization of the 3-D shape by CG technique, whose processing time mainly depends on the number of triangles of 3-D shape reconstructed. Therefore, processing time of the video generation, i.e., the frame rate, is not constant. To solve this problem, we propose a new method which flexibly varies the space resolution of the 3D-shape reconstruction stage, and which makes the processing time of freeviewpoint video nearly constant. We have implemented our method on a PC cluster and realized real-time processing of the free-viewpoint video generation, whose experimental results show the effectiveness of our method.